To add a song from the media library to the queue tap-and-hold on the item's name and select "Add to queue" from the pop-up menu that appears. You can also access the pop-up menu by selecting an item using the navigation keys, then pressing and holding the action or center button until the menu appears.
If a song in the queue is already playing or paused you can quickly add a song to the end of the queue by double-tapping on it within the media library. If the media player is currently stopped double-tapping on a song will clear the current queue and start playing the new song.
Adding a collection to the queue
You can also add a collection of songs at once by using the tap-and-hold menu on a group name instead of just an individual song. For example, if you selected "Add to queue" from the pop-up menu for an artist's name all songs by that artist would be placed in the queue.
The order in which groups of songs are added is context-sensitive based on the group added. For example, if an album is added those songs will be added in the order they appear on the album. If all songs by a particular artist are added those songs will be added in alphabetical order.
Adding a playlist to the queue
The contents of a playlist can also be appended to the queue using the tap-and-hold menu. Your playlists can be found under Music > Playlists, selecting "Add to queue" for any of these will add all items within that playlist to the queue.